1. Field of the Invention
This invention relates to motion dependent video signal processing, and in particular to methods of deriving motion vectors representing motion between fields or frames of a video signal.
2. Description of the Prior Art
Motion dependent video signal processing systems are known in which motion is detected in areas of the image by comparing the contents of blocks of pixels in two successive progressive scan format frames of an input video signal. The position of best correlation of the contents so compared for each block in the first frame indicates the likely motion of the content of that block between the two frames. This motion is represented by a motion vector, and motion vectors derived in this way can be used in subsequent processing stages.
An example of such a process is described in detail in UK patent applications numbers GB-A-2231752 and GB-A-2231746, the contents of these being incorporated herein by reference. GB-A-2231752 and GB-A2231746 both disclose a video standards converter in which fields or frames of a standards converted output video signal are temporally interpolated between pairs of progressive scan input frames using motion vectors derived by comparing the contents of pairs of input frames as previously mentioned. Briefly, the motion vectors are derived as follows.
The content of a block of pixels in a first progressive scan frame is compared with the content of an area in the following progressive scan frame, and a correlation surface is generated which represents the difference in the contents so compared for all possible locations of the block of pixels from the first frame in the area of the following frame. The minimum difference value represented by the correlation surface then indicates the location of best correlation. This minimum value is subjected to a threshold test to determine whether the minimum differs from the next smallest minimum by mope than a predetermined threshold. If a clear minimum, ie one satisfying the threshold test, is identified, the location of this minimum determines the value of a motion vector for the original block representing the motion of the content of that block between the two frames. If a clear minimum is not present, then the correlation surface is grown by adding together the elements of that correlation surface and correlation surfaces for neighbouring blocks, and the grown correlation surface is re-tested for a clear minimum. This process is repeated until either a good motion vector, ie a motion vector determined by a minimum which satisfies the threshold test, is located or certain growth limitations are reached, for example that the original correlation surface has been grown a predetermined number of times. In the example given, there are five levels of growth of correlation surfaces in that a grown correlation surface can be produced for a given block by summing the ungrown correlation surfaces for: a horizontal line of 3.times.1 blocks centered on the given block; a vertical linen of 1.times.3 blocks centered on the given block; a horizontal line of 5.times.1blocks centered on the given block; a 3.times.3 array of blocks centered on the given block; and a 5.times.3 array of blocks centered on the given block.
In another embodiment, all possible grown correlation surfaces for a given block, within the growth limits, are produced, each of the correlation surfaces are tested for a clear minimum, and the best motion vector is then selected. In either embodiment, if no good motion vectors are located, the best available motion vector is passed on to the subsequent processing stages.
The growing of correlation surfaces serves to emphasise common minima in correlation surfaces for neighbouring blocks and thus increases the likelihood that a minimum passing the threshold test will be detected. In many cases, the motion of the contents of adjacent blocks of pixels between two frames will be the same so that the emphasis of common minima resulting from growing of correlation surfaces increases the likelihood that the motion vector derived for a particular block will be the correct one in these cases.
It has been found that using motion vectors derived from the "largest" grown correlation surfaces (ie those produced from 5.times.3 arrays of ungrown surfaces in the example given above) gives subjectively better results for most programme material. However, problems have been observed in some cases, for example where small objects are moving against a background. Due to the averaging effect of growing correlation surfaces, minima appearing in only one, or common to only some, of the correlation surfaces summed to form a grown correlation surface may be masked in the grown correlation surface. Thus, a motion vector derived from the grown correlation surface may not represent the true motion of a given block but may represent, for example, the most common motion among the blocks from which the grown correlation surface is derived. Thus, errors can be introduced when the image includes a small object moving against a background where the growing allows motion in the background to mask the motion of the small object.